<html>
    <head>
        <meta charset="utf-8">
        <title>动态切换代理 - PAC方法</title>
        <link rel="stylesheet" href="../assets/stylesheets/global.css">
        <link rel="stylesheet" href="../assets/stylesheets/words.css">
        <link rel="stylesheet" href="../assets/stylesheets/monokai.css">
        <link rel="stylesheet" href="../assets/stylesheets/table.css">
        <link rel="shortcut icon" href="../assets/images/favicon.ico" type="image/x-icon">
        <link rel="icon" href="../assets/images/favicon.ico" type="image/x-icon">
        <script>
            // 统计代码
            (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
                (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
                m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
                })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');

            ga('create', 'UA-93231524-1', 'auto');
            ga('send', 'pageview');
        </script>
    </head>
    <body>
        <div id="header">
            <a href="../index.html"><div id="logo">JG</div></a>
        </div>
        <div id="container" class="typo">
            <div id="article">
                <h1>动态切换代理 - PAC方法</h1>
                <h4>Posted November 24, 2015</h4>

                <p>最近协助搭建了企业级翻墙系统， 由于没有现成的公司提供这些， 也没有成套比较成熟的方案(国外人用不着, 国内人不敢用的东西). 所以就自己摸索搭建而且也搭建了， 而且相对来说能控制. 可以参考这里<a href="http://omem.me/2015/11/06/offices-proxy/">企业级翻墙方案</a>. 随着很多用户的使用一台服务器显得力不从心, 所以又买了一台然后Squid ＋ Stunnel方案都配好正常启动了。 </p>

<h2>搞定负载均衡</h2>

<p>PAC文件支持故障转移(比较坑, 这种机制比较坑， 我们基本上是避免采取的),比较头疼的是负载均衡, 又两种方案</p>

<ul>
<li>一种是通过负载均衡程序转发</li>
<li>通过随机PAC文件配置.</li>
</ul>

<p>前者的故障转移不好做, 因为客户端代理用的stunnel端口转发到squid端口, 如果stunnel端口依然存活而squid端口还在的话， 这种是转移不了的。然后PAC的故障转移坑的点也在这里. 所以只能不做采取. 后者是我们采取的方案， 开发一个web服务提供pac文件, 然后response的逻辑改下, 随机选择代理服务器生成pac. 这种方法测试一段时间之后能达到我们预期的效果.</p>

<p>负载均衡基于flask服务代码</p>

<p><strong>server.py</strong></p>
<div class="code-wrapper"><span class="lang-label">Python</span><div class="highlight"><pre><span></span><span class="c1"># -*- coding: utf-8 -*-</span>
<span class="sd">&#39;&#39;&#39;</span>
<span class="sd">File Name: web.py</span>
<span class="sd">Author: JackeyGao</span>
<span class="sd">Created Time: 五 11/13 16:11:23 2015</span>
<span class="sd">&#39;&#39;&#39;</span>
<span class="kn">import</span> <span class="nn">random</span><span class="o">,</span> <span class="nn">os</span><span class="o">,</span> <span class="nn">json</span>
<span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">Flask</span>
<span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">make_response</span>
<span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">render_template</span>
<span class="kn">from</span> <span class="nn">flask</span> <span class="kn">import</span> <span class="n">request</span>

<span class="n">app</span> <span class="o">=</span> <span class="n">Flask</span><span class="p">(</span><span class="vm">__name__</span><span class="p">)</span>

<span class="nd">@app.route</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">home</span><span class="p">():</span>
    <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s1">&#39;proxys.json&#39;</span><span class="p">,</span> <span class="s1">&#39;r&#39;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
        <span class="n">proxy_policy</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">())</span>
    <span class="k">return</span> <span class="n">render_template</span><span class="p">(</span><span class="s2">&quot;index.html&quot;</span><span class="p">,</span> <span class="n">count</span><span class="o">=</span><span class="nb">len</span><span class="p">(</span><span class="n">proxy_policy</span><span class="p">))</span>


<span class="nd">@app.route</span><span class="p">(</span><span class="s1">&#39;/proxy.pac&#39;</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">proxy</span><span class="p">():</span>
    <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s1">&#39;proxys.json&#39;</span><span class="p">,</span> <span class="s1">&#39;r&#39;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
        <span class="n">proxy_policy</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">())</span>
    <span class="n">proxys</span> <span class="o">=</span> <span class="p">[</span> <span class="n">proxy</span> <span class="k">for</span> <span class="n">proxy</span><span class="p">,</span> <span class="n">pl</span> <span class="ow">in</span> <span class="n">proxy_policy</span> <span class="k">for</span> <span class="n">n</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">pl</span><span class="p">)</span> <span class="p">]</span>
    <span class="n">random</span><span class="o">.</span><span class="n">shuffle</span><span class="p">(</span><span class="n">proxys</span><span class="p">)</span>
    <span class="n">master</span> <span class="o">=</span> <span class="n">random</span><span class="o">.</span><span class="n">sample</span><span class="p">(</span><span class="n">proxys</span><span class="p">,</span> <span class="mi">1</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span>
    <span class="n">sleve</span>  <span class="o">=</span> <span class="p">[</span> <span class="n">proxy</span> <span class="k">for</span> <span class="n">proxy</span> <span class="ow">in</span> <span class="n">proxys</span> <span class="k">if</span> <span class="n">proxy</span> <span class="o">&lt;&gt;</span> <span class="n">master</span> <span class="p">]</span>
    <span class="k">if</span> <span class="n">sleve</span><span class="p">:</span>
        <span class="n">proxy_list</span> <span class="o">=</span> <span class="s2">&quot;PROXY </span><span class="si">%s</span><span class="s2">;PROXY </span><span class="si">%s</span><span class="s2">;&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">master</span><span class="p">,</span> <span class="n">sleve</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="n">proxy_list</span> <span class="o">=</span> <span class="s2">&quot;PROXY </span><span class="si">%s</span><span class="s2">;&quot;</span> <span class="o">%</span> <span class="n">master</span>

    <span class="n">response</span> <span class="o">=</span> <span class="n">make_response</span><span class="p">(</span><span class="n">render_template</span><span class="p">(</span><span class="s1">&#39;proxy.pac&#39;</span><span class="p">,</span>
        <span class="n">proxy_list</span><span class="o">=</span><span class="n">proxy_list</span><span class="p">))</span>
    <span class="n">response</span><span class="o">.</span><span class="n">mimetype</span> <span class="o">=</span> <span class="s2">&quot;text/plain&quot;</span>
    <span class="k">return</span> <span class="n">response</span>


<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
    <span class="n">port</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">os</span><span class="o">.</span><span class="n">environ</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;PORT&quot;</span><span class="p">,</span> <span class="mi">5000</span><span class="p">))</span>
    <span class="n">app</span><span class="o">.</span><span class="n">run</span><span class="p">(</span><span class="n">host</span><span class="o">=</span><span class="s1">&#39;0.0.0.0&#39;</span><span class="p">,</span> <span class="n">port</span><span class="o">=</span><span class="n">port</span><span class="p">,</span> <span class="n">debug</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
</pre></div>
</div>
<h2>故障转移</h2>

<p>前面已经提到PAC的故障转移是通过Stunnel端口的联通性作出判断的, 当stunnel端口存活而上游的squid端口不通的情况是不会自动转移的。 所以需要一个监控脚本去刷新整个过程的联通性然后把结果给上面的flask web pac服务使用. 这里使用一个reload.py 脚本搞定， 然后把reload.py 做成计划任务.</p>

<p><strong>reload.py</strong></p>
<div class="code-wrapper"><span class="lang-label">Python</span><div class="highlight"><pre><span></span><span class="c1"># -*- coding: utf-8 -*-</span>
<span class="sd">&#39;&#39;&#39;</span>
<span class="sd">File Name: reload.py</span>
<span class="sd">Author: JackeyGao</span>
<span class="sd">Created Time: 二 11/24 13:09:20 2015</span>
<span class="sd">&#39;&#39;&#39;</span>
<span class="kn">import</span> <span class="nn">requests</span>
<span class="kn">import</span> <span class="nn">json</span>

<span class="n">proxy_configs</span> <span class="o">=</span> <span class="p">{</span>
    <span class="s1">&#39;114.xxx.xx.xx:7072&#39;</span><span class="p">:</span> <span class="p">{</span>
        <span class="s1">&#39;user&#39;</span><span class="p">:</span> <span class="s1">&#39;xxxxxx&#39;</span><span class="p">,</span>
        <span class="s1">&#39;passwd&#39;</span><span class="p">:</span> <span class="s1">&#39;xxxxxxx&#39;</span><span class="p">,</span>
        <span class="s1">&#39;return&#39;</span><span class="p">:</span> <span class="s1">&#39;127.0.0.1&#39;</span><span class="p">,</span>
        <span class="s1">&#39;priority&#39;</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
        <span class="p">},</span>
    <span class="s1">&#39;114.xxx.xx.xx:7071&#39;</span><span class="p">:</span> <span class="p">{</span>
        <span class="s1">&#39;user&#39;</span><span class="p">:</span> <span class="s1">&#39;xxxxxx&#39;</span><span class="p">,</span>
        <span class="s1">&#39;passwd&#39;</span><span class="p">:</span> <span class="s1">&#39;xxxxxxxxx&#39;</span><span class="p">,</span>
        <span class="s1">&#39;return&#39;</span><span class="p">:</span> <span class="s1">&#39;127.0.0.1&#39;</span><span class="p">,</span>
        <span class="s1">&#39;priority&#39;</span><span class="p">:</span> <span class="mi">1</span><span class="p">,</span>
        <span class="p">},</span>
    <span class="p">}</span>

<span class="k">def</span> <span class="nf">request_error_handler</span><span class="p">(</span><span class="n">func</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">_deco</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">):</span>
        <span class="k">try</span><span class="p">:</span>
            <span class="k">return</span> <span class="n">func</span><span class="p">(</span><span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kwargs</span><span class="p">)</span>
        <span class="k">except</span> <span class="n">requests</span><span class="o">.</span><span class="n">exceptions</span><span class="o">.</span><span class="n">ConnectionError</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
            <span class="k">if</span> <span class="s1">&#39;407&#39;</span> <span class="ow">in</span> <span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">):</span>
                <span class="k">print</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">%s</span><span class="s2">代理服务器认证失败&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">args</span><span class="p">,))</span>
            <span class="k">else</span><span class="p">:</span>
                <span class="k">print</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">%s</span><span class="s2">链接的其他错误E:(</span><span class="si">%s</span><span class="s2">)&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)))</span>
        <span class="k">except</span> <span class="ne">Exception</span> <span class="k">as</span> <span class="n">e</span><span class="p">:</span>
            <span class="k">print</span><span class="p">(</span><span class="s2">&quot;</span><span class="si">%s</span><span class="s2">发生未知错误E:(</span><span class="si">%s</span><span class="s2">)&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">args</span><span class="p">,</span> <span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)))</span>
        <span class="k">return</span> <span class="bp">False</span>
    <span class="k">return</span> <span class="n">_deco</span>


<span class="nd">@request_error_handler</span>
<span class="k">def</span> <span class="nf">test_proxy_connection</span><span class="p">(</span><span class="n">proxy</span><span class="p">,</span> <span class="n">user</span><span class="p">,</span> <span class="n">passwd</span><span class="p">,</span> <span class="n">return_ip</span><span class="p">):</span>
    <span class="n">proxies</span> <span class="o">=</span> <span class="p">{</span>
        <span class="s2">&quot;https&quot;</span><span class="p">:</span> <span class="s2">&quot;http://</span><span class="si">%s</span><span class="s2">:</span><span class="si">%s</span><span class="s2">@</span><span class="si">%s</span><span class="s2">&quot;</span> <span class="o">%</span> <span class="p">(</span><span class="n">user</span><span class="p">,</span> <span class="n">passwd</span><span class="p">,</span> <span class="n">proxy</span><span class="p">),</span>
    <span class="p">}</span>
    <span class="n">url</span> <span class="o">=</span> <span class="s2">&quot;https://httpbin.org/ip&quot;</span>
    <span class="n">r</span> <span class="o">=</span> <span class="n">requests</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">proxies</span><span class="o">=</span><span class="n">proxies</span><span class="p">,</span> <span class="n">timeout</span><span class="o">=</span><span class="mi">10</span><span class="p">)</span>
    <span class="n">origin</span> <span class="o">=</span> <span class="n">r</span><span class="o">.</span><span class="n">json</span><span class="p">()</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;origin&quot;</span><span class="p">,</span> <span class="bp">None</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">return_ip</span> <span class="o">==</span> <span class="n">origin</span><span class="p">:</span>
        <span class="k">return</span> <span class="bp">True</span>
    <span class="k">else</span><span class="p">:</span>
        <span class="k">return</span> <span class="bp">False</span>


<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
    <span class="n">proxys</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="k">for</span> <span class="n">proxy</span><span class="p">,</span> <span class="n">config</span> <span class="ow">in</span> <span class="n">proxy_configs</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
        <span class="n">status</span> <span class="o">=</span> <span class="n">test_proxy_connection</span><span class="p">(</span><span class="n">proxy</span><span class="p">,</span> <span class="n">config</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;user&#39;</span><span class="p">),</span>
                <span class="n">config</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;passwd&#39;</span><span class="p">),</span> <span class="n">config</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s1">&#39;return&#39;</span><span class="p">))</span>
        <span class="k">if</span> <span class="n">status</span><span class="p">:</span>
            <span class="n">proxys</span><span class="o">.</span><span class="n">append</span><span class="p">((</span><span class="n">proxy</span><span class="p">,</span> <span class="n">config</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s2">&quot;priority&quot;</span><span class="p">)))</span>

    <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s1">&#39;proxys.json&#39;</span><span class="p">,</span> <span class="s1">&#39;w&#39;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
        <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="n">json</span><span class="o">.</span><span class="n">dumps</span><span class="p">(</span><span class="n">proxys</span><span class="p">))</span>
</pre></div>
</div>
<p>然后经过pac文件路径改成现在起的地址.需要说明一点有些系统支持的pac文件每隔一段时间会重载pac文件, 这个时间越快对于客户端的故障转移就越及时. pac服务端的故障转移取决于reload.py 的执行间隔, 可以在crontab里面设置为5分钟, 甚至更少的时间.</p>

            </div>
            <div id="footer">
                <a href="../words.html"><div id="more-words">MORE WORDS</div></a>
            </div>
        </div>
    </body>
</html>